Seminar given by Jim Crutchfield of the Complexity Sciences Center, UC Davis on October 15, 2008 for the Redwood Center for Theoretical Neuroscience at UC Berkeley.
Abstract. I will show how theory building can naturally distinguish between regularity and randomness. Starting from basic modeling principles, using rate distortion theory and computational mechanics I'll argue for a general information-theoretic objective function that embodies a trade-off between a model's complexity and its predictive power. The family of solutions derived from this principle corresponds to a hierarchy of models. At each level of complexity, they achieve maximal predictive power, identifying a process's exact causal organization in the limit of optimal prediction. Examples show how theory building can profit from analyzing a process's causal compressibility, which is reflected in the optimal models' rate-distortion curve.
